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INTRODUCTION 


There  Is  a  great  desire  for  engineers  and  scientists  to  model  the  processing 
functions  of  the  nervous  system.  Some  feel  that  these  models  should  be  directly 
related  to  the  actual  system,  so  they  study  the  neuron,  the  primary  element  of  the 
nervous  system.  These  Individuals  model  their  neural  networks  after  studies  per¬ 
formed  on  the  nervous  system  of  humans  and  other  organisms.  Others  feel  that  since 
complete  knowledge  of  the  nervous  system  is  very  far  away,  the  neural  networks 
should  be  modeled  as  black  boxes,  and  the  transfer  functions  of  these  “boxes”  are 
derived  mathematically,  Either  way,  neural  networks  allow  for  parallel  processing  of 
information  that  can  greatly  reduce  the  time  required  to  perform  operations.  This  is 
very  important  in  pattern  recognition. 


DISCUSSION 


The  Neuron 

The  neuron  is  the  basic  component  of  the  nervous  system  of  any  organism.  The 
neurons,  placed  together,  form  the  nerves.  A  neuron  (fig.  1 )  is  a  single  cell  composed 
of  a  cell  body  or  soma,  dendrites  for  receiving  information,  and  an  axon  for  transmitting 
the  signal.  The  dendrites  extend  out  to  the  other  neurons  and  cells  and  act  as 
processors  of  the  signals  coming  from  these  cells.  These  dendrites  conduct  the 
incoming  signals  to  the  soma  without  any  amplification  of  the  signal. 

The  axon  carries  the  signal  out  of  the  soma  to  the  next  cell.  The  signal  is 
transmitted  through  the  axon  similar  to  the  fashion  in  which  a  signal  is  conveyed 
through  an  electric  cable.  Just  as  an  electric  cable  often  has  an  insulative  coating,  so 
do  the  axons.  This  insulation  on  the  axon  is  call  myelin  sheath.  The  myelin  is 
attached  to  the  axon  leaving  gaps  in  the  material  where  the  bare  axon  is  exposed. 
These  gaps  are  called  Nodes  of  Ranvier.  The  electric  signal  is  transferred  from  one 
Node  of  Ranvier  to  another  which  is  referred  to  as  saltatory  conduction.  This  transmis¬ 
sion  is  much  quicker  than  on  an  unmyelinated  neuron. 

There  are  two  ways  that  the  signal  can  be  transferred  from  one  cell  to  another- 
electrically  or  chemically.  An  electrical  signal  can  be  directly  transferred  from  one  cell 
to  another  if  the  cells  are  very  close  together.  This  is  called  a  tight  junction.  A 
chemical  transfer  is  a  bit  me  complicated,  yet  more  common.  It  occurs  over  a  gap 
between  the  cells  called  the  synapse.  When  the  electric  signal  Is  transmitted  down 
the  axon,  a  chemical  is  released  from  the  presynaptic  side  into  the  gap.  This  chemical 
is  received  on  the  postsynaptlc  side  of  the  gap  which  in  turn  generates  a  new  signal. 

When  an  action  potential  reaches  the  end  of  a  neuron  axon,  the  signal  must 
cross  over  a  small  gap,  called  the  synaptic  gap,  to  reach  the  next  neuron  dendrite. 
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There  are  certain  chemicals  at  the  presynaptlc  side  called  neurotranomitters.  These 
chemicals  are  released  Into  the  gap  and  received  by  neuroreceptors  at  the  postsynap- 
tic  side.  The  type  of  chemical  used  as  the  neurotransmitter  is  considered  either 
excitatory  or  inhibitory.  The  normal  resting  potential  of  the  neuron  is  -70  mV.  If  the 
transmitter  Is  excitatory,  the  potential  will  be  raised  about  100  mV  to  +30  mV.  Howev¬ 
er,  an  Inhibitory  transmitter  will  cause  a  drop  In  the  potential  about  10  mV  to  -80  mV. 

The  electrical  signal  that  travels  through  the  neuron  is  generated  by  concentra¬ 
tion  gradients  between  the  Ions  Inside  and  outside  of  the  cell.  An  ion  Is  a  chemical 
which  Is  either  missing  or  has  extra  electrons,  giving  it  a  positive  or  negative  charge. 
The  major  Ions  of  Interest  In  any  organism  are  sodium  (Na+),  chloride  (Cl*),  and 
potassium  (K+).  If  a  chemical  of  one  concentration  Is  separated  from  the  same 
chemical  of  a  different  concentration  by  a  thin  membrane  with  pores  large  enough  for 
the  chemical  to  pass,  diffusion  will  take  place.  The  chemical  on  the  side  with  the 
greater  concentration  will  diffuse  across  the  membrane  to  equalize  the  concentration 
between  the  two  sides.  This  Is  diffusion  by  a  concentration  gradient.  If  there  Is  a 
charge  gradient  (i.e.,  one  side  has  a  more  positive  charge  than  the  other),  diffusion  will 
take  place  to  equalize  the  charges.  To  make  things  more  complicated,  there  is  a 
concentration  gradient  as  well  as  a  charge  gradient  In  the  neuron  cell.  This  movement 
of  chemicals,  which  is  also  a  movement  of  charges  across  the  cell  wall,  cause  the 
potential  difference  between  the  Internal  and  external  fluids. 

When  a  neuron  soma  receives  a  signal,  there  Is  a  threshold  voltage  that  the 
potential  of  the  neuron  must  reach  In  order  to  fire  an  action  potential  or  an  impulse. 
This  is  an  all-or-none  response  in  that  if  this  threshold  Is  not  met,  there  will  be  no 
signal,  but  If  this  limit  is  reached,  there  will  be  a  signal  to  a  specific  voltage.  This  action 
potential  is  always  the  same  voltage  level  for  each  neuron,  no  matter  how  high  the 
input  voltage  Is.  The  input  voltage  is  comprised  of  all  the  inputs  from  the  dendrites, 
However,  depending  of  the  lengths  of  the  dendrites,  these  signals  can  be  delayed 
from  one  another. 

Neuronal  Modeling 

As  technology  advances,  engineers  are  trying  to  design  and  Improve  models  of 
the  action  potential  and  the  interaction  of  the  neurons.  These  models  are  being  used 
to  simulate  processes  that  are  common  In  living  organisms.  When  these  processes 
become  advanced,  they  start  to  be  more  realistic  In  terms  of  modeling  the  human 
reasoning  operation.  These  models  of  neuronal  systems  are  called  artificial  neural 
networks  (ANN)  or  neural  nets. 

There  Is  some  controversy  between  the  developers  of  these  neural  networks. 
Some  investigators  feel  that  the  networks  should  be  modeled  after  the  actual  neurons 
in  the  human  body.  This  is  difficult  because  although  Individual  neurons  can  be 
studied,  as  often  is  with  the  squid  giant  axon,  people  do  not  have  a  full  grasp  on  how 
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the  neurons  Interact.  The  other  school  of  thought  is  to  treat  the  neuronal  system  as  a 
black  box.  The  input  and  the  output  are  known,  but  the  Inner  steps  are  the  question. 
These  neural  net  developers  make  their  own  relationships  to  correlate  the  output  with 
the  Input. 

The  most  Important  fact  about  both  types  of  modeling  Is  that  many  operations 
can  be  done  at  the  same  time.  This  parallel  processing  Is  the  basis  for  using  neural 
networks.  Most  artificial  computational  devices  process  Information  In  series.  For  an 
elementary  example,  if  one  were  to  calculate  the  mean  of  10  numbers  (fig.  2),  the  sum 
of  two  numbers  would  be  found,  then  the  sum  with  the  next  number,  and  so  on.  After 
nine  operations,  the  sum  of  all  10  numbers  is  obtained,  then  divide  by  10.  This  takes 
10  computational  steps.  In  a  parallel  processor,  the  first  two  numbers  would  be  added 
together  at  the  same  time  as  the  second  two,  and  so  on  (fig.  3).  Therefore,  five 
operations  would  take  place  during  the  time  it  takes  to  do  one  computation.  The  next 
operation  would  be  again  adding  the  first  two  and  the  second  two.  This  would 
continue  until  all  of  the  numbers  were  added  which  would  take  four  steps;  the  division 
would  be  the  fifth  step.  In  this  simple  example,  the  series  processor  took  ten  steps 
when  the  parallel  processor  only  took  five.  The  parallel  processor  takes  fewer  steps 
and  therefore  less  time  to  perform  the  desired  task.  In  a  much  more  difficult  computa¬ 
tion,  this  difference  would  have  been  greater.  The  use  of  neural  networks  can  greatly 
decrease  the  speed  of  operation. 

Basic  Neural  Networks 

The  most  basic  model  of  a  neuron  is  the  McCulloch-PItts  neuron  (fig.  4),  named 
after  the  creators.  This  model  is  similar  to  the  electrical  analog  operational  amplifier  or 
op  amp.  There  is  a  finite  number  of  inputs  (X|)  into  each  neuron,  each  with  its  own 
weighting  coefficient  (W|).  In  the  simple  McCulloch-Pitts  neuron  these  weights  are  +1 
for  an  excitatory  neuron  and  -1  for  an  inhibitory  neuron.  These  weights  are  multiplied 
by  the  inputs  to  the  neuron.  There  is  one  output  (y)  for  each  neuron.  The  neuron  is 
given  a  threshold  value  (L).  If  the  sum  of  all  the  weighted  Inputs  reaches  that  threshold 
value,  then  the  neuron  will  fire;  that  Is,  the  output  of  the  neuron  will  be  high.  However, 
if  the  sum  of  the  weighted  inputs  does  not  meet  the  threshold,  the  neuron  will  not  fire, 
or  the  output  will  be  low.  The  output  of  the  neuron  can  be  represented  by  the  equation 

y-o(Z>iX,-i) 

where  g(p)  is  the  function  such  that  g(p)  ■  0  if  p  <  0  and  g(p)  ■  +1  if  p  >  0.  If  a  number 
of  neurons  are  put  together  in  a  network,  then  the  output  for  the  j th  neuron  would  be 
written  as 
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yi  -  9  (SwijXij  -  lj) 

Each  of  the  neurons  In  the  network  will  perform  this  operation  simultaneously  at 
discrete  moments  In  time. 

A  simple  example  is  the  logical  AND  gate  (fig.  5).  This  neural  net  has  one 
neuron  with  two  excitatory  inputs  and  a  threshold  value  of  two.  If  both  inputs  are  low  or 
zeros,  the  sum  will  be  zero  and  the  threshold  will  not  be  met.  Therefore,  the  output  will 
be  zero.  If  one  of  the  inputs  Is  zero  while  the  other  Is  one,  then  the  sum  will  be  one, 
which  still  is  not  enough  to  make  the  neuron  fire.  Finally  if  both  inputs  are  high,  the 
sum  will  be  two.  This  reaches  the  threshold  value  causing  the  neuron  to  fire  or  the 
output  to  be  one. 

A  logical  OR  gate  (fig.  6)  is  very  similar  to  the  AND  gate.  There  are  again  two 
excitatory  inputs,  but  this  time  the  threshold  value  Is  one.  A  logical  NOT  or  inverter  (fig, 
7)  has  a  single  inhibitory  neuron  with  a  threshold  of  zero.  These  are  all  simple  neural 
nets  consisting  of  only  one  neuron. 

The  EXCLUSIVE  OR  gate  (fig.  8)  consists  of  five  neurons  in  the  network.  The 
two  inputs  go  into  inverters  where  the  outputs  then  go  to  AND  gates.  The  outputs  of 
these  gates  go  into  an  OR  gate.  This  is  called  a  three  layer  network.  In  a  biological 
system,  the  first  layer,  in  this  case  it  is  the  inverters,  is  called  the  sensory  layer.  This 
layer  first  processes  the  information  from  the  outside  world.  This  could  be  a  pressure 
sensor  or  a  tactile  sensor  in  the  body.  The  final  layer,  in  this  case  the  OR  gate,  is 
called  the  motor  layer.  This  is  the  motor  command  to  move  the  part  of  the  body.  For 
example,  if  a  person’s  finger  were  to  feel  something  hot,  then  this  hot  substance  would 
be  recognized  in  the  sensory  layer.  This  would  cause  the  person  to  move  the  finger 
away  from  that  object.  This  muscle  movement  comes  from  the  motor  layer.  All  other 
systems  in  the  body  respond  this  way  too.  Suppose,  for  example,  that  a  baroreceptor, 
or  pressure  sensor,  in  an  artery  measures  the  blood  pressure  to  be  too  high.  The 
motor  functions  that  respond  this  would  reduce  the  heart  rate  at  the  sino-atrial  (SA) 
node  by  decreasing  the  firing  frequency;  this  would  in  turn  cause  an  increase  In  the 
filtration  rate  at  the  kidneys,  All  of  the  processing  that  goes  on  between  the  sensing 
and  the  movement  are  in  the  hidden  layers.  In  the  EXCLUSIVE  OR  gate,  there  is  only 
one  hidden  layer,  which  is  the  AND  gate  layer.  An  artificial  neural  network  has  inputs 
into  a  sensory  layer;  goes  through  any  number  of  hidden  layers;  and  then  to  the  motor 
layer. 

More  Advanced  Neural  Networks 

The  basic  difference  between  the  natural  neuronal  system  and  the  artificial 
McCulloch-Pitts  neuronal  model  is  that  a  natural  system  shows  some  form  of  learning. 
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In  visual  evoked  potential  (VEP)  experiments  conducted  by  the  author  on  human 
subjects,  an  Image  was  shown  for  60  sec  followed  by  a  rest  period.  This  was  contin¬ 
ued  over  10  cycles.  During  this  time,  an  electroencephalogram  (EEQ)  was  taken  at 
the  visual  cortex  of  the  brain.  The  first  reading  had  a  peak  that  was  about  20  pV  higher 
than  a  flat,  black  image,  During  each  successive  viewing  of  the  Image,  the  peak 
voltage  measured  was  decreased.  When  a  different  image  was  presented  at  the  end 
of  the  experiment,  the  voltage  jumped  up  roughly  20  4V  from  the  previous  peak.  This 
peak  was  higher  than  the  peak  of  the  first  Image.  This  shows  that  the  combination  of 
neurons  does  not  always  fire  at  a  certain  voltage.  As  the  pattern  was  learned,  the 
response  became  less  because  the  weights  on  the  neurons  decreased.  When  a  new 
pattern  had  to  be  relearned,  the  weights  Increased.  The  weights  on  a  natural  neuron 
are  not  always  +1  or  -1.  This  is  how  the  learning  process  takes  place.  In  most  artificial 
neural  networks,  the  weights  can  be  changed  through  feedback  techniques. 

There  are  many  different  methods  that  people  have  used  to  adjust  the  weights. 
For  example,  D.  0.  Hebb  calculates  the  change  In  weighting  coefficient  to  be  propor¬ 
tional  to  the  output  of  the  neuron*.  S.  Gossberg  uses  a  relationship  where  the  change 
in  weighting  coefficient  is  proportional  to  the  difference  between  the  output  and  the 
weighting  coefficient*.  Both  of  these  methods  are  based  on  biological  principles.  A 
nonblological  method  of  calculating  the  change  in  the  weighting  coefficient  Is  by  an 
amount  proportional  to  the  partial  derivative  of  the  error  with  respect  to  the  weight. 
This  error  requires  that  a  desired  value  Is  known.  This  model  uses  back  propagation 
to  calculate  the  partial  derivative.  This  Is  a  truly  mathematical  method,  since  neuronal 
synapses  do  not  reverse  direction,  There  are,  however,  many  different  methods  used 
to  calculate  the  weighting  coefficient, 


Applications  of  Artificial  Neural  Networks 

The  fact  that  processes  performed  In  parallel  with  neural  networks  can  greatly 
reduce  the  time  required  to  perform  the  operation  is  what  makes  the  use  of  the  neural 
nets  so  appealing.  Neural  nets  can  be  used  to  quickly  analyze  real-time  data  that  is 
being  collected.  An  important  use  of  neural  computing  is  In  pattern  recognition. 

The  United  States  Postal  Service  is  beginning  to  use  pattern  recognition  to  read 
written  addresses.  In  most  cases,  each  address  must  be  personally  read  and  sorted 
by  humans.  Now  some  systems  are  being  developed  to  find  the  nlne-dlglt  zip  code 
on  the  address,  read  it,  and  sort  it  accordingly.  The  difficulty  with  this  Is  that  people  do 


*Vemuri,V„  "Artificial  Neural  Networks:  An  Introduction,"  Artificial  Neural  Networks: 

Ihflfliallfial  Concepts,  pp  1-12, 1 988. 
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not  always  use  a  nine-digit  zip  code;  they  put  the  zip  code  in  different  places  on  the 
address  (after  the  state  or  under  the  city)  and  write  numbers  differently.  Therefore, 
many  different  forms  of  each  number  must  be  recognized  accurately.  Parallel 
processing  is  used  when  the  number  must  be  compared  to  all  of  the  stored  templates 
of  the  different  numbers  and  the  different  forms  of  each  number. 

Another  form  of  pattern  recognition  that  is  encountered  is  by  the  military.  A 
precision  guided  munition  must  somehow  be  directed  to  the  correct  target.  Usually 
this  is  done  simplistically  with  a  series  processor.  However,  as  seen  in  Operation 
Desert  Storm,  there  was  a  big  problem  with  friendly  fire;  that  is,  our  munitions  sensing 
our  own  vehicles  as  targets.  There  is  work  being  performed  on  neurocomputers  to 
develop  an  accuracy  in  identifying  types  of  targets.  For  example,  the  TRW  Mark  IV 
neurocomputer  can  identify  a  type  of  aircraft  with  95%  accuracy.  The  input  image  of 
an  airplane  is  taken  from  directly  over  the  plane  with  the  plane  in  any  angle.  These 
relative  intensities  throughout  the  image  are  digitized  into  pixels.  A  neural  network 
takes  each  pixel  and  calculates  the  polar  coordinates  based  on  the  origin  being  at  the 
center  of  the  image.  This  processed  image  is  sent  to  another  neural  network  that 
recognizes  and  classifies  the  image  independent  of  the  angle.  This  technology  is  just 
starting  to  emerge  and  there  is  still  a  long  way  to  go. 


CONCLUSIONS 

Artificial  neural  networks  are  being  used  to  approximately  simulate  the  process¬ 
ing  that  takes  place  in  the  natural  nervous  system.  Although  we  are  very  far  from 
simulating  the  actual  nervous  system,  certain  aspects  of  the  system  can  be  modeled 
with  some  degree  of  accuracy.  The  visual  system  is  often  the  system  used.  For 
example,  the  human  visual  system  can  sense  something  and  distinguish  it  very 
quickly.  A  person  can  quickly  look  at  another  person  from  many  different  angles,  in 
different  clothing,  and  different  hair  styles,  and  still  be  able  to  determine  who  that 
person  is.  A  model  of  this  processing  would  have  to  have  many  different  people’s 
images,  and  different  forms  of  these  images  stored  in  memory.  The  detected  image  of 
the  person  would  have  to  compared  to  each  of  the  images  stored.  This  would  take 
longer  and  with  less  accuracy  than  the  human  processing  system.  Much  work  has 
been  done  in  the  artificial  neural  network  technology,  but  we  are  a  long  way  off  from 
mastering  the  natural  neural  network  of  humans. 
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Figure  2.  Series  calculation  of  the  mean  of  ten  inputs  (takes  ten  steps) 
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Figure  3.  Parallel  calculation  of  mean  of  ten  inputs  (takes  five  steps) 


divide  — 


8 


NOTE:  There  are  n  inputs  (XI ,  X2,  X3...Xn);  each  with  an  assigned  weighting  coeffi¬ 
cient  (wl ,  w2,  23. ,.wn);  a  threshold  value  of  L,  and  an  output  Y. 


Figure  4.  McCulloch-Pitts  model  of  the  neuron 


9 


c.  truth  table 


Figure  5.  Logical  AND  gate 
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Figure  6.  Logical  OR  gate 
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a.  electronic  symbol  b.  neuronal  model  X 
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Figure  7.  Inverter 
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Figure  8.  Logical  exclusive  OR  gate 
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